A Metric for Evaluating Discourse Coherence based on Coreference Resolution
نویسندگان
چکیده
We propose a simple and effective metric for automatically evaluating discourse coherence of a text using the outputs of a coreference resolution model. According to the idea that a writer tends to appropriately utilise coreference relations when writing a coherent text, we introduce a metric of discourse coherence based on automatically identified coreference relations. We empirically evaluated our metric by comparing it to the entity grid modelling by Barzilay and Lapata (2008) using Japanese newspaper articles as a target data set. The results indicate that our metric better reflects discourse coherence of texts than the existing model.
منابع مشابه
Corpus based coreference resolution for Farsi text
"Coreference resolution" or "finding all expressions that refer to the same entity" in a text, is one of the important requirements in natural language processing. Two words are coreference when both refer to a single entity in the text or the real world. So the main task of coreference resolution systems is to identify terms that refer to a unique entity. A coreference resolution tool could be...
متن کاملCoreference-inspired Coherence Modeling
Research on coreference resolution and summarization has modeled the way entities are realized as concrete phrases in discourse. In particular there exist models of the noun phrase syntax used for discourse-new versus discourse-old referents, and models describing the likely distance between a pronoun and its antecedent. However, models of discourse coherence, as applied to information ordering...
متن کاملKnowledge-Lean Coreference Resolution And Its Relation To Textual Cohesion And Coherence
In this paper we present a new empirical method for coreference resolution, implemented in the COCKTAIL system. The resuits of COCKTAIL are used for lightweight abduction of cohesion and coherence structures. We show that referential cohesion can be integrated with lexical cohesion to produce pragmatic knowledge. Upon this knowledge coherence abduction takes place. I M o t i v a t i o n Corefer...
متن کاملOn Coreference Resolution Performance Metrics
The paper proposes a Constrained EntityAlignment F-Measure (CEAF) for evaluating coreference resolution. The metric is computed by aligning reference and system entities (or coreference chains) with the constraint that a system (reference) entity is aligned with at most one reference (system) entity. We show that the best alignment is a maximum bipartite matching problem which can be solved by ...
متن کاملThe Life and Death of Discourse Entities: Identifying Singleton Mentions
A discourse typically involves numerous entities, but few are mentioned more than once. Distinguishing discourse entities that die out after just one mention (singletons) from those that lead longer lives (coreferent) would benefit NLP applications such as coreference resolution, protagonist identification, topic modeling, and discourse coherence. We build a logistic regression model for predic...
متن کامل